1. Field of the Invention
The present invention relates to an apparatus for transmitting and receiving stream data which forms a three-dimensional virtual space represented by three-dimensional computer graphics (hereinafter, referred to as three-dimensional CG or 3DCG) based on a network such as the Internet.
2. Description of the Related Art
In recent years, as the fields to which 3DCG is applicable, a virtual mall, electric commerce, and various home pages related thereto on the Internet such as a WWW (World Wide Web) have drawn attention. In particular, with the rapid development of the Internet, an environment is becoming available for easily handling 3DCG with a relatively high quality such as games and movies at home. In the conventional WWW, a machine called a server such as a personal computer and a work station is connected to a plurality of machines called clients such as personal computers. When a client requests, the client downloads data such as information of an image, a voice, a text, and an arrangement provided by the server, if required. The data is reconstructed on the client side, whereby a user can obtain required information. A communication method based on a TCP/IP (Transmission Control/Internet Protocol) is adopted for communication between the server and the client.
Conventionally, data provided by the server is mainly text data and image data. Recently, with the advancement in standardization of VRML (Virtual Reality Modeling Language), a browser of VRML, and the like, there is a tendency of transferring 3DCG data such as a shape and/or a scene.
Hereinafter, the VRML will be briefly described.
According to a conventional data format such as an image and a text as in HTML (Hyper Text Markup Language), a great amount of transfer time and transfer cost are required for transferring image data, in particular, animation data. Therefore, in the current system, there is a limit to the network traffic. In contrast, according to conventional 3DCG, visual point information and light source information including configurations are all processed by using three-dimensional data obtained by CG (Computer Graphics) data by calculation. On the receiving side, image data is generated based on three-dimensional data, and it is displayed.
With the advancement in CG technology, the quality of an image created by CG is rapidly improved. Therefore, it is becoming very efficient to transfer CG data as it is in view of a data amount.
Typically, when CG data is used, the data can be compressed at a data compression ratio of 1/100 or less, compared with the case where comparable image data is transferred. Thus, there is a tendency of standardizing a method for transferring 3DCG data through a network. As one solution, it is proposed to standardize 3 DCG data called VRML (VRML Ver. 2.0). In VRML Ver. 2.0, formats of data such as data having a configuration called a primitive, light source data, visual point data, and texture data, and a method for specifying a movement of a rigid body.
On the other hand, there is an animation technique for generating an image in real time, which has drawn attention recently in the CG field. By using an animation technique in real time, it is attempted to reproduce a real movement of a CG character particularly in CMs and movies. As an example, a complicated shape of a human is represented by a skeleton structure, and a motion amount of joints of a skeleton which change from moment to moment is defined, whereby a complicated motion can be naturally reproduced.
However, according to the conventional 3DCG modeling language on the Internet based on the VRML, a motion cannot be set in real time to a structure having a complicated shape such as a human.
Furthermore, it is impossible to transmit and receive motion data of a CG character (e.g., human) which performs real motions in real time through a network with a narrow band such as a telephone line.
A transmitter of the present invention includes a multi-dimensional stream data transmitting unit, for transmitting multi-dimensional stream data.
In one embodiment of the present invention, the transmitter includes: a multi-dimensional stream data identifying unit for classifying input data into a plurality of kinds of motion data; and a multi-dimensional stream data compressing unit for compressing the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit changes a compression ratio in accordance with the kind of the motion data.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit selects a compression method in accordance with the kind of the motion data.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit generates a node sequence which is time-series data, composed of at least one motion data among the plurality of kinds of motion data during a predetermined period, and extracts an arbitrary node from the node sequence to compress the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data transmitting unit transmits interpolation information required for reproducing the extracted node and the node sequence.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit reduces a high frequency component of the plurality of kinds of motion data to compress the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit subjects at least one motion data among the plurality of kinds of motion data to coordinate transformation to compress the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit transforms data represented by three-dimensional vector coordinates among the plurality of kinds of motion data to data represented by polar coordinates, thereby compressing the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data includes header information or stream format describing data, describing a kind of the multi-dimensional stream data to be transmitted, an order of the plurality of kinds of motion data to be transmitted, and a length of the plurality of kinds of motion data to be transmitted.
In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.
In another embodiment of the present invention, the space information of a plurality of dimensions includes position information, rotation information, and scale information.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene includes data representing a position, rotation and a scale of the CG scene.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.
In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels and a header portion describing a stream data format of the data portion and, each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.
In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the at least one joint.
In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.
In another embodiment of the present invention, the above-mentioned transmitter includes a multi-dimensional stream data retaining unit for retaining data compressed by the multi-dimensional stream data compressing unit in a multi-hierarchical structure, wherein the transmitter selects data retained in a certain hierarchical structure from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which a receiver receiving the multi-dimensional stream data reproduces the multi-dimensional stream data, and transmits the selected data.
In another embodiment of the present invention, the above-mentioned transmitter selects data at a hierarchy level, at which a reproducible data amount is the largest, from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted, and a reproduction speed at which the receiver receiving the multi-dimensional stream data reproduces the multi-dimensional stream data.
In another embodiment of the present invention, the above-mentioned transmitter further includes a multi-dimensional stream data retaining unit for retaining data compressed by the multi-dimensional stream data compressing unit in a file format of the multi-dimensional stream data.
According to another aspect of the present invention, a transmitting method includes a multi-dimensional stream data transmitting step of transmitting multi-dimensional stream data.
In one embodiment of the present invention, the above-mentioned transmitting method includes: a multi-dimensional stream data identifying step of classifying input data into a plurality of kinds of motion data; and a multi-dimensional stream data compressing step of compressing the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit changes a compression ratio in accordance with the kind of the motion data.
In another embodiment of the present invention, the multi-dimensional stream data compressing unit selects a compression method in accordance with the kind of the motion data.
In another embodiment of the present invention, in the multi-dimensional stream data compressing step, a node sequence is generated, which is time-series data, composed of at least one motion data among the plurality of kinds of motion data during a predetermined period, and extracts an arbitrary node from the node sequence to compress the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the above-mentioned transmitting method includes the step of transmitting interpolation information required for reproducing the extracted node and the node sequence.
In another embodiment of the present invention, in the multi-dimensional stream data compressing step, a high frequency component of the plurality of kinds of motion data is reduced to compress the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, in the multi-dimensional stream data compressing step, at least one motion data among the plurality of kinds of motion data is subjected to coordinate transformation to compress the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, in the multi-dimensional stream data compressing step, data represented by three-dimensional vector coordinates among the plurality of kinds of motion data is transformed to data represented by polar coordinates, thereby compressing the plurality of kinds of motion data on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data includes header information or stream format describing data, describing a kind of the multi-dimensional stream data to be transmitted, an order of the plurality of kinds of motion data to be transmitted, and a length of the plurality of kinds of motion data to be transmitted.
In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.
In another embodiment of the present invention, the space information of a plurality of dimensions includes position information, rotation information, and scale information.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene includes data representing a position, rotation and a scale of the CG scene.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.
In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels, and a header portion describing a stream data format of the data portion, and each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.
In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the plurality of joints.
In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.
In another embodiment of the present invention, the above-mentioned transmitting method includes a multi-dimensional stream data retaining step of retaining data compressed by the multi-dimensional stream data compressing unit in a multi-hierarchical structure, wherein the transmitting method selects data retained in a certain hierarchical structure from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed for reproducing the multi-dimensional stream data, and transmits the selected data.
In another embodiment of the present invention, data at a hierarchy level, at which a reproducible data amount is the largest, is selected from the data retained in the multi-hierarchical structure, in accordance with a state of a transmission path through which the multi-dimensional stream data is to be transmitted, and a reproduction speed at which the receiver receiving the multi-dimensional stream data reproduces the multi-dimensional stream data.
In another embodiment of the present invention, the above-mentioned transmitting method further includes a multi-dimensional stream data retaining step of retaining data compressed in the multi-dimensional stream data compressing step in a file format of the multi-dimensional stream data.
According to another aspect of the present invention, a receiver includes a multi-dimensional stream data receiving unit for receiving multi-dimensional stream data.
In one embodiment of the present invention, the receiver includes: a multi-dimensional stream data identifying unit for identifying a data content of the multi-dimensional stream data from one of header information and stream format describing information of the multi-dimensional stream data; a multi-buffering unit for sequentially reading the multi-dimensional stream data into a plurality of multi-buffers on a block basis; and a multi-dimensional stream data decompressing unit for decompressing the multi-dimensional put in blocks on a block basis.
In another embodiment of the present invention, the multi-dimensional stream data identifying unit determines whether or not the received multi-dimensional stream data is a channel definition packet.
In another embodiment of the present invention, the multi-dimensional stream data identifying unit determines whether or not the received multi-dimensional stream data is a data packet.
In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a channel definition packet, the multi-dimensional stream data identifying unit identifies a time stamp, a total number of channels, and a compression ratio identifier representing a compression ratio of channel data from the received multi-dimensional stream data.
In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a data packet, the multi-dimensional stream data identifying unit identifies a time stamp and a total number of channels from the received multi-dimensional stream data.
In another embodiment of the present invention, the received multi-dimensional stream data contains compressed motion data and compression information.
In another embodiment of the present invention, the multi-dimensional stream data decompressing unit decompresses the compressed motion data based on the compression information.
In another embodiment of the present invention, the multi-dimensional stream data decompressing unit determines a decompression ratio based on the compression information.
In another embodiment of the present invention, the multi-dimensional stream data decompressing unit determines a decompression method based on the compression information.
In another embodiment of the present invention, in the case where the compressed motion data is generated by extracting arbitrary motion data from a plurality of motion data corresponding to a predetermined period, the multi-dimensional stream data decompressing unit recovers a plurality of motion data corresponding to the predetermined period based the extracted motion data and the compression information.
In another embodiment of the present invention, the compression information is interpolation information.
In another embodiment of the present invention, the multi-dimensional stream data decompressing unit decompresses the compressed motion data by subjecting the compressed motion data to coordinate transformation based on the compression information.
In another embodiment of the present invention, in the case where the compressed motion data is represented by polar coordinates, the multi-dimensional stream data decompressing unit decompresses the compressed motion data by converting the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates.
In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.
In another embodiment of the present invention, the space information of a plurality of dimensions has position information, rotation information, and scale information.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene has data representing a position, a rotation, and a scale of the CG scene.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.
In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels and a header portion describing a stream data format of the data portion, and each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.
In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the plurality of joints.
In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.
In another embodiment of the present invention, the receiver indicates a compression ratio of the multi-dimensional stream data to a transmitter for transmitting the multi-dimensional stream data, in accordance with at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed at which the receiver reproduces the multi-dimensional stream data.
According to another aspect of the present invention, a receiving method includes a multi-dimensional stream data receiving step of receiving multi-dimensional stream data.
In one embodiment of the present invention, the above-mentioned receiving method includes: a multi-dimensional stream data identifying step of identifying a data content of the multi-dimensional stream data from one of header information and stream format describing information of the multi-dimensional stream data; a multi-buffering step of sequentially reading the multi-dimensional stream data into a plurality of multi-buffers on a block basis; and a multi-dimensional stream data decompressing step of decompressing the multi-dimensional put in blocks on a block basis.
In one embodiment of the present invention, in the multi-dimensional stream data identifying step, it is determined whether or not the received multi-dimensional stream data is a channel definition packet.
In another embodiment of the present invention, in the multi-dimensional stream data identifying step, it is determined whether or not the received multi-dimensional stream data is a data packet.
In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a channel definition packet, in the multi-dimensional stream data identifying step, a time stamp, a total number of channels, and a compression ratio identifier representing a compression ratio of channel data are identified from the received multi-dimensional stream data.
In another embodiment of the present invention, in the case where the multi-dimensional stream data identifying unit determines that the received multi-dimensional stream data is a data packet, in the multi-dimensional stream data identifying step, a time stamp and a total number of channels are identified from the received multi-dimensional stream data.
In another embodiment of the present invention, the received multi-dimensional stream data contains compressed motion data and compression information.
In another embodiment of the present invention, the multi-dimensional stream data decompressing unit decompresses the compressed motion data based on the compression information.
In another embodiment of the present invention, in the multi-dimensional stream data decompressing step, a decompression ratio is determined based on the compression information.
In another embodiment of the present invention, in the multi-dimensional stream data decompressing step, a decompression method is determined based on the compression information.
In another embodiment of the present invention, in the case where the compressed motion data is generated by extracting arbitrary motion data from a plurality of motion data corresponding to a predetermined period, in the multi-dimensional stream data decompressing step, a plurality of motion data corresponding to the predetermined period are recovered based the extracted motion data and the compression information.
In another embodiment of the present invention, the compression information is interpolation information.
In another embodiment of the present invention, in the multi-dimensional stream data decompressing step, the compressed motion data is decompressed by subjecting the compressed motion data to coordinate transformation based on the compression information.
In another embodiment of the present invention, in the case where the compressed motion data is represented by polar coordinates, in the multi-dimensional stream data decompressing step, the compressed motion data is decompressed by converting the compressed motion data represented by polar coordinates into data represented by three-dimensional vector coordinates.
In another embodiment of the present invention, the multi-dimensional stream data has time information and space information of a plurality of dimensions.
In another embodiment of the present invention, the space information of a plurality of dimensions has position information, rotation information, and scale information.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene has data representing a position, a rotation, and a scale of the CG scene.
In another embodiment of the present invention, the multi-dimensional stream data capable of representing a CG scene is multi-dimensional animation stream data.
In another embodiment of the present invention, the multi-dimensional stream data has a data portion having a plurality of channels and a header portion describing a stream data format of the data portion and, each of the plurality of channels has a plurality of pieces of information regarding a three-dimensional CG character at a certain time.
In another embodiment of the present invention, the three-dimensional CG character is capable of being represented by a hierarchical structure having at least one joint, and each of the plurality of motion data has at least one of data having a dimension of a position vector at a reference position of the three-dimensional CG character, data having a dimension of a direction vector representing a posture at the reference position, data having a dimension of an angle vector of the at least one joint and a rotation angle around the angle vector, and stream data having a dimension of a rotation angle around a local coordinates axis of the at least one joint.
In another embodiment of the present invention, the multi-dimensional stream data is a packet having a header portion and a data portion, and the header portion represents a kind of the packet, an order of the packet, and a length of the data portion.
In another embodiment of the present invention, a compression ratio of the multi-dimensional stream data is indicated to a transmitter for transmitting the multi-dimensional stream data, in accordance with at least one of a state of a transmission path through which the multi-dimensional stream data is to be transmitted and a reproduction speed for reproducing the multi-dimensional stream data.
In another embodiment of the present invention, the multi-dimensional stream data contains motion data and voice data, the motion data has time stamp information, and the voice data has time stamp information, and the receiver reproduces the motion data and the voice data in synchronization, in accordance with the time stamp information of the motion data and the time stamp information of the voice data.
In another embodiment of the present invention, the multi-dimensional stream data contains motion data and voice data, the motion data has time stamp information, and the voice data has time stamp information, and the receiving method further comprises the step of reproducing the motion data and the voice data in synchronization, in accordance with the time stamp information of the motion data and the time stamp information of the voice data.
According to another aspect of the present invention, a recording medium capable of being read by a computer is provided, in which a program for allowing the computer to execute the above-mentioned transmitting method is recorded.
According to another aspect of the present invention, a recording medium capable of being read by a computer is provided, in which a program for allowing the computer to execute the above-mentioned receiving method is recorded.
According to another aspect of the present invention, a recording medium capable of being read by a computer is provided, in which a program for allowing the computer to execute the above-mentioned receiving method is recorded.
According to another aspect of the present invention, a communication system includes: a transmitter for transmitting multi-dimensional stream data; and a receiver for receiving the multi-dimensional stream data.
In one embodiment of the present invention, the transmitter includes one of an artificial satellite, a ground wave broadcasting station, and a wire broadcasting station, and the receiver is one of a digital TV and a set top box.
In the above-mentioned transmitter, the multi-dimensional stream data contains data capable of representing a CG scene.
In the above-mentioned transmitting method, the multi-dimensional stream data contains data capable of representing a CG scene.
In the above-mentioned receiver, the multi-dimensional stream data contains data capable of representing a CG scene.
In the above-mentioned receiving method, the multi-dimensional stream data contains data capable of representing a CG scene.
In the above-mentioned communication system, the multi-dimensional stream data contains data capable of representing a CG scene.
Thus, the invention described herein makes possible the advantage of providing an apparatus for transmitting and receiving multi-dimensional stream data, which is capable of transmitting and receiving motion data of a CG character in real time.
This and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.